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ABSTRACT 

At Virginia Highlands Community College (VHCC) , 
institutional assessment activities were used to evaluate the 
effectiveness of the Academic and Instructional Support Services 
(AISS) division, a non-academic unit which includes the Library for 
archival/retrieval purposes, the Learning Lab for the testing and 
learning center, and Instructional Support Services which provides 
resources related to information technology. Sources of information 
for the assessment included the AISS advisory coajnittee, surveys of 
student use and opinions, the academic computing committee, and an 
external evaluation. In institutional effectiveness efforts at VHCC, 
non-academic departments are not held accountable for student 
"failure" or "success," but only for having a process in place for 
stating objectives, measuring accomplishments, and using results to 
improve programming. To maintain quality services, the AISS division 
adheres to the following objectives: (1) establish a clearly defined 
purpose; (2) formulate educational goals consistent with the 
institution's purpose; (3) develop qualitative and quantitative 
procedures for evaluating the extent to which these goals are being 
achieved; and (4) use the results of these evaluations to improve 
services and operations. The assessment identified several problem 
areas, including a lack of central planning for academic computing; 
too many workers reporting directly to one supervisor; an overlap of 
similar tasks; an<J instructors uncertain of the services available at 
different areas of the AISS, A description of the stages in the 
development of AISS's new academic computing policy, a flexible 
design methodology called Educational Information Architecture, is 
provided. (PAA) 
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Changing the organizational structure of 
Nonacadexnic Departments 
for Institutional Effectiveness 



INTRODUCTION 



AISS MISSION 

The Academic and Instructional Support Services (AISS) division is the learning resources unit of the 
college. The AISS unit constitutes the Library for archival/retrieval purposes, Learning Lab for a testing 
and learning center, and Instructional Support Services which provides information technology services. 
The main mission of this unit is to support instruction. 

Purpose : 

The efforts of faculty and staff will demonstrate how institutional effectiveness results were used to 
restructure a nonacademic department, Academic and Instructional Support Services (AISS) for an 
effective support of the teaching/learning process and the institution as a whole. 

Description: 

Utilizing assessment the following problems were identified for institutional effectiveness: 

- lack of central planning for academic computing 

- too many workers reporting directly to one supervisor 

- overlapping of tasks, similar in nature 

- instructors uncertain of which area to request services 

Research, surveys, and reports indicated colleges were considering the merging of media and computer 
services because of the nature of their technology. Laser technology, multimedia, P C computers, LAN 
systems, interactive learning, and a host of high-tech invents have all given rise to new concepts in the 
teaching/learning process for educational technology. Instructional technology has required that instructors 
re-think the practice they use to present material in class. Prepared instructional material reduces 30% of 
class time required to disseminate and record the information. M We simply must recognize how outdated 
our current teaching practices are - through optimum use of technology in teaching and learning" 
(Roueche, 1993). 

Assessment activities were kept simple without high levels of statistical or research expertise required. 
The following represent the means used for the assessment of the effectiveness of this new department: 

A. Advisory committee 

B. Opinion survey 

C. Academic computing committee 

D. Personal contact survey 

E. External evaluation 

The critical factor was the use of the data collected for departmental improvement. 



Rather than attempt to explain assessment or institutional effectiveness, this presentation is to convey the 
basic level of information necessary to initiate procedures for effectiveness and to describe to some extent 
the transformation that takes place organizationally. Qualitative and quantitative means were used to 
assess the department. Qualitative describes evaluations of holistic judgements and quantitative 
characterizes the identification of individual components and provision of a quantitative score. 

The Academic and Instructional Support Service division was restructured for institutional effectiveness. 
An assessment was conducted which identified changes that needed to be made within the AISS division. 
The current staff was used to redefine areas of responsibility. The Instructional Support Services (ISS) 
department was the result of this reorganization within the division. The creation of this department was 
a amalgamation of instructional media services with academic computing. The merging of these two 
technology services provides a more efficient means to serve educational technology at the college. 

In preparation for reaffirmation of regional accreditation administrative and educational support, 
departments are expected to provide evidence that their operations support accomplishments of their 
institution's Expanded Statement of Institutional Purpose by accomplishing their departmental intended 
goals. Virginia Highlands Community College's (VHCC) focus was on the educational "process" as 
directed by the Virginia Community College System (VCCS)in the area of teaching/learning. Since the 
1960s there has been an emerging shift in focus from teaching to learning (Roueche, 1993). Within 
institutional effectiveness, this linkage to and support of the institution's purpose is absolutely essential. 
For this reason the ISS department has clear objectives which are curriculum based. 

Good research is the key to informed decision-making. Research is more a process than a product, a 
process of accumulating evidence that supports or fails to support a particular theory. The "process" rather 
than student "outcomes" were used to establish administrative objectives for the nonacademic department. 
Within institutional effectiveness, nonacademic departments are not held accountable for "failure" or 
"success", only for having in place a process for stating objectives, measuring accomplishments, and using 
the results to improve programming. Administrative objectives focus on the results of educational support 
operations rather than the processes themselves. To enhance the process, Instructional Support Services 
established results-oriented statements regarding its operations. Focus upon "process" and the impact of 
these efforts are stated in results-oriented terms, whenever possible. For example, instead of stating that 
we will improve the computer access for CIS students, a results-oriented administrative objective would 
state, "all CIS students have 100% access during the normal operation hours of the college to computers 
for classroom instruction or college related use. 

The reaffirmation of accreditation under institutional effectiveness is the broadening of the statements of 
purpose to recognize the need for educational support. Virginia Highland's statement of purpose is not 
only a philosophical but a functional one. Once administrative objectives are identified, a means for 
assessment of their accomplishment is developed. The means of assessment commonly identified with 
nonacademic departments include clientele satisfaction or attitude and direct measures or counts of 
departmental survey as well as specific services. The attitudinal questionnaire is a standardized survey 
that provides normative response patterns. These attitudes are gauged by a detailed questionnaire 
regarding services in a composite survey of the Learning Lab and Instructional Support Services 
departments. The composite survey serves to reduce the number of times faculty and students are 
requested to respond to surveys, preventing repetitive research. Utilization of standardized attitudinal 
surveys provide standardized items for normative response patterns. 



Demings states that nearly 90 percent of all organizational problems are management problems in colleges 
(Letarte, 1993). In developing a model for Instructional Support Services the long-term approach and not 
a quick-fix will be the key. Total Quality Management (TQM), Total Quality Learning (TQL), 
Assessment, and Institutional Effectiveness are means that represent a significant restructuring of how an 
organization thinks and acts. A college wide system for establishing shared vision, mission, goals, and 
methods for focused improvement actions must involve all of these concepts. The result is the creation 
of a value-based organization. In order for this to happen a paradigm shift - a willingness to break with 
old ideas and develop a mental openness - must occur. 

The newly created Instructional Support Services initiated a strategic plan for a comprehensive 
instructional technology effectiveness initiative. Assessment is defined as the systematic evaluation of how 
well that technology serves the college in relation to its goals and mission. Assessment plays a key role 
to determine the effectiveness of educational technology at the college. When assessing the effectiveness 
of instructional technology, we compare the intended goals against the results achieved in the stated 
mission of the college. Instructional technology is curricular based and not something that materializes 
on the campus to simply implement new emerging technologies. The college believes instruction is 
number one. Instructional Support Services responds to the needs of the faculty, students and staff and 
keeps realistic goals in line with the college mission. 

COMPONlgNreg OT * PTAMMTMa AMD KVALUATION PROCESS 
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Reprinud by permission of author 

The level of quality depends on the college's successful use of process and resources to achieve established 
goals. Continuous study, analysis and appraisal of the college's purposes, policies, procedures and 
programs are necessary to use resources efficiently In order to accomplish this task Instructional Support 
Services adheres to four simple statements outlined : 



1 . Establish a clearly defined purpose. 

PURPOSE: meet the technology demands for educational technology of the college which 
are linked to the curriculum. 

2. Formulate educational goals consistent with the institution's purpose. 

GOAL: improve, enhance, and extend instruction through information. Implementation 
of total quality service and a commitment to shared decision making. Develop resources, 
fctaff, and facilities for the most effective utilization of the teaching/learning process for 
specific student outcomes. 

3. pevelop procedures for evaluating the extent to which these educational goals are being 
achieved. 

Indicators of effectiveness are in place at the college and their supporting measurement 
standards provide quantitative and qualitative results. 

Qualitative Instruments 

a. Implement computing policy, EIA developed 

b. Academic Computing Committee 

c. Advisory committee (AISS) 

d. Personal contact survey 

e. Opinion survey 

f. Assessment of the College's Master Plan, state academic computing survey, SCHEV & SACS 
reports. 

g. External evaluation 

h. 30 minute management staff meetings 

Quantitative Instruments 

a. Assessment of the effectiveness of interactive video for Allied Health courses. 

b. Business/Industry survey for CIS courses, software, and hardware. 

c. Graph, charts, and statistic figures for total output for the department. 

4. Use. of the results of these evaluations to improve institutional services and operations. 
Formative and Summative assessment processes for Instructional Support Services are used to 
determine goals and objectives and effectiveness of the department. The above information is all 
used to determine the direction and continuous change the department must implement in order 
to support educational technology effectively (Nichols, 1991). 

ISS recognizes that all students and faculty should have access to adequate learning resources needed to 
support its purposes and programs. To support its curriculum, the college provides a variety of facilities 
and instructional support services. We provide a electronic performance support system which integrates 
information, tools, and methodology for the user. The goal is to provide whatever is necessary to generate 
performance and learning at the moment of need. These programs are organized and administered so as 
to provide easy access for its users, which are adequate to support the educational process, and contribute 
to the effectiveness of learning and efficient use of resources. Instructional technology services are 
essential to educational programs and are available to support the college's management and administrative 
functions as well. Technology now demands integration of method and content; students, learning through 
technology , are not only learning new content, but they are learning the new technology (Roueche, 1993). 
Policies for the allocation of these resources must be clearly stated and consistent with the college's 
purpose and its goals. 
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Site-based management is used to reform the instructional support for the college. Developing specific 
strategies for Instructional Support Services has produced a quality-focused organization. Making 
observations about behavior and priorities to develop "customer satisfaction" for redefining and improving 
the educational quality fen* instruction. Assertive leadership is a catalyst for developing broad-based 
consensus among faculty. 

Traditional staff development in instructional strategies and skills was not enough to bring about the 
quality of changes necessary to restructure the division. Strategies for making fundamental changes in 
services should aim to discover, develop, and focus the attitudes and talents of staff around a broad-based 
meaningful vision of the goals of Instructional Support Services. It is the attitudes and beliefs that guide 
behavior, so therefore we must change what people believe is their role and responsibility in the process. 
The design for a comprehensive initiative was called for to use information to guide both policy and 
practice of Instructional Support Services. In the final analysis the responsibility by faculty are the 
determining factors for success. 

This is a systematic approach with consistently based data and other objective measures. A focus on 
process rather than outcome measures is used to document policy. Appropriate process measures focus 
allows accountability and effectiveness to be assessed according to criteria that can be established, in part, 
by faculty, administrators, and other constituents. The following is an example of a new implemented for 
academic computing: 

ACADEMIC COMPUTING POLICY 

Educational Information Architecture (EIA) is a flexible design methodology to help education build 
systems that can respond to the challenges of technology. A primary objective of a information resources 
management system is to identify and satisfy user needs. EIA is a framework for designing and creating 
computing systems that attempt to inspire a common vision for those who participate in the process and 
produce a thorough analysis and avoid confusion. A critical way of assessing the technology and 
information needs of the college is outlined in this plan/and to articulate the vision and communicate it 
to the institution. 

The use of LANs has produced client/server technology. It is these applications and networks for an wen 
systems solution which pose options in terms of facilitating the engineering of systems. EIA has the 
potential to be an effective approach to systems design; and the flexibility to meet the ever-changing 
information needs of a comprehensive community college. An advantage is the flexibility to assess the 
impact of change by identifying significant interactions and connections of the architecture. It 
encompasses aspects of the college to create a vision and framework for systems design. 

EIA provides a vision and framework for future information systems. In addition it provides a transition 
plan on how to achieve goals and objectives. First determine the objectives to guide the transition plan. 
Based on the transition objectives and the EIA vision the best strategy for the transition will be 
determined. As potential changes arise, the impact on the specific stage will be assessed. Modifications 
should be made in increments. A formal process is pursued to assess the impact on all the stages of the 
development. By progressing outward from the stages all ihe critical impacts can be predicted and a path 
to move forward identified. 



STAPES FO JR DgygLOPMENT OF AN EIA: 



1. Conception 

2. Definition 

3. Design 

4. Implementation 

5. Testing 

6. Translation 



QftgATXNP AN EIA: 

There are six stages for creating an Educational Information Architecture, 
qnvi ronmtnt 

1. The goal is to identify strategies and concerns around which the systems strategy 
can be built. 

Ob-iectiv 

2. The department will perform a current systems review and meet with users to 
identify their requirements. 

Data Architecture 

3. A technology-independent model depicting the college's data requirements of entity 
types, attribute types, relations types, and integrity constraints. The high-level model 
could identify where data overlap may or may not occur. 

Applicati on Architecture 

4. The department needs to identify where the processes and data will be located and 
how the various processes will work together. 

infra, lit rue t ur e 

5. Identify all the infrastructure needs looking at how data and applications might be 
shared by different departments and offices in the college. 

6. Because the operating system software and database software etc. is shared, it is 
necessary to identify system software that meets the needs application within the 
constraints of the system software. Hardware also falls in this category to identify the 
mainframe, modems, microcomputers, work stations, and LANs which support the 
applications, and data requirements of the college. 



Our surveys indicate that students enjoy using technology and they expect to have technology 
incorporated into their learning. Though its effect is often difficult to measure, we are convinced 
technology plays an important role in learning. 

In a future analysis, Instructional Support Services is developing a process to further provide 
information for assessment by engaging in research to prove that technology affects instruction. Our 
belief is supported by EDCOM's president Robert Heterrick, who says, "we're going to focus our 
limited resources on trying to provide some very hard experimental evidence that the fusion of 
information technology into the teaching/learning process provides both better and more effective 
learning, and does it in cost parameters that are better than those that we typically do today" (Wilson, 
1993). The Instructional Support Services department will make every effort to secure and provide 
information about the effectiveness of instructional technology concerning the teaching/learning 
process. National and in-house information determines the effectiveness of instruction and provides a 
high standard for excellence. 
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